Update MiniMax-M3 SPEED-Bench AL collector to EAGLE3-GQA head#2290
Update MiniMax-M3 SPEED-Bench AL collector to EAGLE3-GQA head#2290qiching wants to merge 3 commits into
Conversation
Switch the MiniMax-M3 golden-AL collector from the original MHA head (Inferact/MiniMax-M3-EAGLE3) to the new GQA head (Inferact/MiniMax-M3-EAGLE3-GQA), which supersedes it (num_key_value_heads=4, 16x smaller draft KV cache). DRAFT_MODEL is now env-overridable so the NVFP4 variant can be collected without editing the script. FLASH_ATTN is retained (works for GQA; backend choice does not affect the measured AL). The golden AL table (golden_al_distribution/minimaxm3_eagle3.yaml) is regenerated by the speedbench-al CI, not hand-edited.
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
AL is a per-draft acceptance property invariant to concurrency, so batch prompts to cut wall-clock. Kept at 16 (the max-concurrency the GQA head was characterized at) to avoid vLLM auto-disabling spec decode under large batches.
AL is a per-draft acceptance property invariant to concurrency, so batch prompts to cut wall-clock. Kept at 16 (the max-concurrency the GQA head was characterized at) to avoid vLLM auto-disabling spec decode under large batches.
|
could you paste AL values ? cc @functionstackx |
2 participants
Switch the MiniMax-M3 golden-AL collector from the original MHA head (Inferact/MiniMax-M3-EAGLE3) to the new GQA head
(Inferact/MiniMax-M3-EAGLE3-GQA), which supersedes it (num_key_value_heads=4, 16x smaller draft KV cache). DRAFT_MODEL is now env-overridable so the NVFP4 variant can be collected without editing the script. FLASH_ATTN is retained (works for GQA; backend choice does not affect the measured AL). The golden AL table (golden_al_distribution/minimaxm3_eagle3.yaml) is regenerated by the speedbench-al CI, not hand-edited.